System and method for tailoring an electronic digital assistant inquiry response as a function of previously detected user ingestion of related video information

ABSTRACT

A process at an electronic computing device that tailors an electronic digital assistant generated inquiry response as a function of previously detected user ingestion of related information includes receiving, from a video capture device configured to track a gaze direction of a first user, a video stream including a first field-of-view of the first user. An object is then identified in the video stream first field-of-view remaining in the first field-of-view for a determined threshold period of time, and the object processed via a video processing algorithm to produce object information, which is then stored. Subsequently, an inquiry is received from the first user for information, and it is determined that the inquiry is related to the object information. The electronic digital assistant then provides a response to the inquiry as a function of the object information.

BACKGROUND OF THE INVENTION

Tablets, laptops, phones (e.g., cellular or satellite), mobile(vehicular) or portable (personal) two-way radios, and other mobilecomputing devices are now in common use by users, such as firstresponders (including firemen, police officers, and paramedics, amongothers), and provide such users and others with instant access toincreasingly valuable additional information and resources such asvehicle histories, arrest records, outstanding warrants, healthinformation, real-time traffic or other situational status information,and any other information that may aid the user in making a moreinformed determination of an action to take or how to resolve asituation, among other possibilities.

Many such mobile computing devices further comprise, or provide accessto, electronic digital assistants (or sometimes referenced as “virtualpartners”) that can provide the user thereof with valuable informationin an automated (e.g., without further user input) or semi-automated(e.g., with some further user input) fashion. The valuable informationprovided to the user can be based on explicit requests for suchinformation posed by the user via an input (e.g., such as a parsednatural language input or an electronic touch interface manipulationassociated with an explicit request) in which the electronic digitalassistant may reactively provide such requested valuable information, orcan be based on some other set of one or more context or triggers inwhich the electronic digital assistant may proactively provide suchvaluable information to the user absent any explicit request from theuser.

As some existing examples, electronic digital assistants such as Siriprovided by Apple, Inc.® and Google Now provided by Google, Inc.®, aresoftware applications running on underlying electronic hardware that arecapable of understanding natural language, and may complete electronictasks in response to user voice inputs, among other additional oralternative types of inputs. These electronic digital assistants mayperform such tasks as taking and storing voice dictation for futurereference and retrieval, reading a received text message or an e-mailmessage aloud, generating a text message or e-mail message reply,looking up requested phone numbers and initiating a phone call to arequested contact, generating calendar appointments and providingappointment reminders, warning users of nearby dangers such as trafficaccidents or environmental hazards, and providing many other types ofinformation in a reactive or proactive manner.

Furthermore, as the number of fixed, body-worn, and vehicle-equippedcameras available to users and accessible across departments andorganizations increases, the opportunity and ability to identify andcategorize objects of interest in real-time and use this information tosupplement electronic digital assistant accessible information improvesas well.

However, while conventional electronic digital assistants may be capableof sourcing these video streams when providing responses to userinquiries, such conventional electronic digital assistants have not beensuccessful in more intelligently filtering such information for formingresponses to user inquiries.

Thus, there exists a need for an improved technical method, device, andsystem for an electronic digital assistant to computationally processvideo relative to a determined user's perception of such video and totailor the electronic digital assistant's inquiry response to asubsequent query from the user accordingly.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

The accompanying figures, where like reference numerals refer toidentical or functionally similar elements throughout the separateviews, which together with the detailed description below areincorporated in and form part of the specification and serve to furtherillustrate various embodiments of concepts that include the claimedinvention, and to explain various principles and advantages of thoseembodiments.

FIG. 1 is a system diagram illustrating a system for operating anelectronic digital assistant, in accordance with some embodiments.

FIG. 2 is a device diagram showing a device structure of an electroniccomputing device for operating an electronic digital assistant, inaccordance with some embodiments.

FIG. 3 illustrates a flowchart setting forth process steps for operatingthe electronic digital assistant of FIGS. 1 and/or 2, in accordance withsome embodiments.

Skilled artisans will appreciate that elements in the figures areillustrated for simplicity and clarity and have not necessarily beendrawn to scale. For example, the dimensions of some of the elements inthe figures may be exaggerated relative to other elements to help toimprove understanding of embodiments of the present invention.

The apparatus and method components have been represented whereappropriate by conventional symbols in the drawings, showing only thosespecific details that are pertinent to understanding the embodiments ofthe present invention so as not to obscure the disclosure with detailsthat will be readily apparent to those of ordinary skill in the arthaving the benefit of the description herein.

DETAILED DESCRIPTION OF THE INVENTION

Disclosed is an improved technical method, device, and system for anelectronic digital assistant to computationally process video relativeto a determined user's perception of such video and to tailor theelectronic digital assistant's inquiry response to a subsequent queryfrom the user accordingly.

In one embodiment, a process for tailoring an electronic digitalassistant generated inquiry response as a function of previouslydetected user ingestion of related information includes: receiving, atan electronic processing device from a video capture device configuredto track a gaze direction of a first user, a video stream including afirst field-of-view substantially matching a field-of-view of the firstuser; identifying, by the electronic processing device, an object in thevideo stream first field-of-view remaining in the first field-of-viewfor a determined threshold period of time and, responsively, processingthe object via a video processing algorithm and causing, by theelectronic processing device, object information regarding the objectoutput from the video processing algorithm to be stored in anon-volatile store; subsequently receiving, at the electronic processingdevice, an inquiry for information from the first user; determining, bythe electronic processing device, that the inquiry is related to theobject information; and providing, by the electronic processing device,a response to the inquiry as a function of the object information.

In a further embodiment, an electronic processing device for tailoringan artificial intelligence inquiry response as a function of previouslydetected user ingestion of related information includes: a memory; atransceiver; and one or more processors configured to: receive, from avideo capture device configured to track a gaze direction of a firstuser, a video stream including a first field-of-view substantiallymatching a field-of-view of the first user; identify an object in thevideo stream first field-of-view remaining in the first field-of-viewfor a determined threshold period of time and, responsively, process theobject via a video processing algorithm and causing, by the electronicprocessing device, object information regarding the object output fromthe video processing algorithm to be stored in a non-volatile store;subsequently receive an inquiry for information from the first user;determine that the inquiry is related to the object information; andprovide a response to the inquiry, via one of the transceiver, a displaycommunicatively coupled to the electronic computing device, or a speakercommunicatively coupled to the electronic computing device, as afunction of the object information.

Each of the above-mentioned embodiments will be discussed in more detailbelow, starting with example communication system and devicearchitectures of the system in which the embodiments may be practiced,followed by an illustration of processing steps for achieving theimproved technical method, device, and system for an electronic digitalassistant to computationally process video relative to a determineduser's perception of such video and to tailor the electronic digitalassistant's inquiry response to a subsequent query from the useraccordingly. Further advantages and features consistent with thisdisclosure will be set forth in the following detailed description, withreference to the figures.

1. Communication System and Device Structures

a. Communication System Structure

Referring now to the drawings, and in particular FIG. 1, a communicationsystem diagram illustrates a system 100 of devices including a first setof devices that a user 102 (illustrated in FIG. 1 as a first responderpolice officer) may wear, such as a primary battery-powered portableradio 104 used for narrowband and/or broadband direct-mode orinfrastructure communications, a battery-powered radio speakermicrophone (RSM) video capture device 106, a laptop 114 having anintegrated video camera and used for data applications such as incidentsupport applications, smart glasses 116 (e.g., which may be virtualreality, augmented reality, or mixed reality glasses, may include avideo camera, and/or may include a head-tracking and/or eye-trackingfunction), sensor-enabled holster 118, and/or biometric sensor wristband120. Although FIG. 1 illustrates only a single user 102 with arespective first set of devices, in other embodiments, the single user102 may include additional sets of same or similar devices, andadditional users may be present with respective additional sets of sameor similar devices.

System 100 may also include a vehicle 132 associated with the user 102(the user 102 illustrated as potentially alternatively sitting in adriver's seat of vehicle 132 in FIG. 1 instead of standing outside ofit) having an integrated vehicular computing device 133, an associatedvehicular video camera 134 and/or microphone 135, a coupled vehiculartransceiver 136, and a head and/or eye-tracking device 139. AlthoughFIG. 1 illustrates only a single vehicle 132 with a respective singlevehicular computing device 133, single vehicular video camera 134 andsingle transceiver 136, in other embodiments, the vehicle 132 mayinclude additional same or similar computing devices, video camerasand/or transceivers, and additional vehicles may be present withrespective additional sets of computing devices, video cameras, and/ortransceivers.

Each of the portable radio 104, RSM video capture device 106, laptop114, and vehicular computing device 133 may be capable of directlywirelessly communicating via direct-mode wireless link(s) 142, and/ormay be capable of wirelessly communicating via a wireless infrastructureradio access network (RAN) 152 over respective wireless link(s) 140, 144and via corresponding transceiver circuits.

The portable radio 104, in particular, may be any mobile computingdevice used for infrastructure RAN or direct-mode media (e.g., voice,audio, video, etc.) communication via a long-range wireless transmitterand/or transceiver that has a transmitter transmit range on the order ofmiles, e.g., 0.5-50 miles, or 3-20 miles (e.g., in comparison to ashort-range transmitter such as a Bluetooth, Zigbee, or NFC transmitter)with other mobile computing devices and/or the infrastructure RAN 152.The long-range transmitter may implement a direct-mode, conventional, ortrunked land mobile radio (LMR) standard or protocol such as ETSIDigital Mobile Radio (DMR), a Project 25 (P25) standard defined by theAssociation of Public Safety Communications Officials International(APCO), Terrestrial Trunked Radio (TETRA), or other LMR radio protocolsor standards. In other embodiments, the long range transmitter mayimplement a Long Term Evolution (LTE), LTE-Advance, or 5G protocolincluding multimedia broadcast multicast services (MBMS) or single sitepoint-to-multipoint (SC-PTM) over which an open mobile alliance (OMA)push to talk (PTT) over cellular (OMA-PoC), a voice over IP (VoIP), anLTE Direct or LTE Device to Device, or a PTT over IP (PoIP) applicationmay be implemented. In still further embodiments, the long rangetransmitter may implement a Wi-Fi protocol perhaps in accordance with anIEEE 802.11 standard (e.g., 802.11a, 802.11b, 802.11g) or a WiMAXprotocol perhaps operating in accordance with an IEEE 802.16 standard.

In the example of FIG. 1, the portable radio 104 may form the hub ofcommunication connectivity for the user 102, through which otheraccessory devices such as a biometric sensor (for example, the biometricsensor wristband 120), an activity tracker, a weapon status sensor (forexample, the sensor-enabled holster 118), a heads-up-display (forexample, the smart glasses 116), the RSM video capture device 106,and/or the laptop 114 may communicatively couple.

In order to communicate with and exchange video, audio, and other mediaand communications with the RSM video capture device 106, laptop 114,and/or smart glasses 116, the portable radio 104 may contain one or morephysical electronic ports (such as a USB port, an Ethernet port, anaudio jack, etc.) for direct electronic coupling with the RSM videocapture device 106, laptop 114, and/or smart glasses 116 and/or maycontain a short-range transmitter (e.g., in comparison to the long-rangetransmitter such as a LMR or Broadband transmitter) and/or transceiverfor wirelessly coupling with the RSM video capture device 106, laptop114, and/or smart glasses 116. The short-range transmitter may be aBluetooth, Zigbee, or NFC transmitter having a transmit range on theorder of 0.01-100 meters, or 0.1-10 meters.

In other embodiments, the RSM video capture device 106, the laptop 114,and/or the smart glasses 116 may contain their own long-rangetransceivers and may communicate with one another and/or with theinfrastructure RAN 152 or vehicular transceiver 136 directly withoutpassing through portable radio 104.

The RSM video capture device 106, in particular, provides voicefunctionality features similar to a traditional RSM, including one ormore of acting as a remote microphone that is closer to the user's 102mouth, providing a remote speaker allowing play back of audio closer tothe user's 102 ear, and including a PTT switch or other type of PTTinput. The voice and/or audio recorded at the remote microphone may beprovided to the portable radio 104 for storage and/or analysis or forfurther transmission to other mobile communication devices or theinfrastructure RAN 152, or may be directly transmitted by the RSM videocapture device 106 to other mobile computing devices or to theinfrastructure RAN 152. The voice and/or audio played back at the remotespeaker may be received from the portable radio 104 or received directlyfrom one or more other mobile computing devices or the infrastructureRAN 152. The RSM video capture device 106 may include a separatephysical PTT switch 108 that functions, in cooperation with the portableradio 104 or on its own, to maintain the portable radio 104 and/or RSMvideo capture device 106 in a monitor only mode, and which switches thedevice(s) to a transmit-only mode (for half-duplex devices) or transmitand receive mode (for full-duplex devices) upon depression or activationof the PTT switch 108. The portable radio 104 and/or RSM video capturedevice 106 may form part of a group communications architecture thatallows a single mobile computing device to communicate with one or moregroup members (not shown) associated with a particular group of devicesat a same time.

Additional features may be provided at the RSM video capture device 106as well. For example, a display screen 110 may be provided fordisplaying images, video, and/or text to the user 102 or to someoneelse. The display screen 110 may be, for example, a liquid crystaldisplay (LCD) screen or an organic light emitting display (OLED) displayscreen. In some embodiments, a touch sensitive input interface may beincorporated into the display screen 110 as well, allowing the user 102to interact with content provided on the display screen 110. A soft PTTinput may also be provided, for example, via such a touch interface.

A video camera 112 may be further provided at the RSM video capturedevice 106, integrating an ability to capture images and/or video andstore the captured image data (for further analysis) or transmit thecaptured image data as an image or video stream to the portable radio104 and/or to other mobile computing devices or to the infrastructureRAN 152 directly. The video camera 112 and RSM remote microphone may beused, for example, for capturing audio and/or video of a field-of-viewassociated with the user 102, storing the captured audio and/or videodata for further analysis or transmitting the captured audio and/orvideo data as an audio and/or video stream to the portable radio 104and/or to other mobile computing devices or to the infrastructure RAN152 directly for further analysis. The RSM remote microphone may be anomni-directional or unidirectional microphone or array ofomni-directional or unidirectional microphones that may be capable ofidentifying a direction from which a captured sound emanated.

In some embodiments, the RSM video capture device 106 may be replacedwith a more limited body worn camera that may include the video camera112 and/or microphone noted above for capturing audio and/or video, butmay forego one or more of the features noted above that transform thebody worn camera into a more full featured RSM, such as the separatephysical PTT switch 108 and the display screen 110, and remotemicrophone functionality for voice communications in cooperation withportable radio 104.

The laptop 114, in particular, may be any wireless computing device usedfor infrastructure RAN or direct-mode media communication via along-range or short-range wireless transmitter with other mobilecomputing devices and/or the infrastructure RAN 152. The laptop 114includes a display screen for displaying a user interface to anoperating system and one or more applications running on the operatingsystem, such as a broadband PTT communications application, a webbrowser application, a vehicle history database application, a workflowapplication, a forms or reporting tool application, an arrest recorddatabase application, an outstanding warrant database application, amapping and/or navigation application, a health information databaseapplication, or other types of applications that may require userinteraction to operate. The laptop 114 display screen may be, forexample, an LCD screen or an OLED display screen. In some embodiments, atouch sensitive input interface may be incorporated into the displayscreen as well, allowing the user 102 to interact with content providedon the display screen. A soft PTT input may also be provided, forexample, via such a touch interface.

Front and/or rear-facing video cameras may be further provided at thelaptop 114, integrating an ability to capture video and/or audio of theuser 102 and/or a field of view substantially matching the user's 102,and store and/or otherwise process the captured video and/or audio forfurther analysis or transmit the captured video and/or audio as a videoand/or audio stream to the portable radio 104, other mobile computingdevices, and/or the infrastructure RAN 152 for further analysis.

The smart glasses 116 may include a digital imaging device, a computingdevice, a short-range and/or long-range transceiver device, and/or aprojecting device. The smart glasses 116 may maintain a bi-directionalcommunications connection with the portable radio 104 and provide analways-on or on-demand video feed pointed in a direction of the user's102 gaze via the digital imaging device, and/or may provide a personaldisplay via the projection device integrated into the smart glasses 116for displaying to its user information such as text, images, or videoreceived from the portable radio 104 or directly from the infrastructureRAN 152. In some embodiments, an additional user interface mechanismsuch as a touch interface or gesture detection mechanism may be providedat the smart glasses 116 that allows the user 102 to interact with thedisplay elements displayed on the smart glasses 116 or projected intothe user's 102 eyes, or to modify operation of the digital imagingdevice, while in other embodiments, a display and input interface at theportable radio 104 may be provided for interacting with smart glasses116 content and modifying operation of the digital imaging device, amongother possibilities.

The smart glasses 116 may provide a virtual reality interface in which acomputer-simulated reality electronically replicates an environment withwhich the user 102 may interact, may provide an augmented realityinterface in which a direct or indirect view of real-world environmentsin which the user is currently disposed are augmented, i.e.,supplemented, by additional computer-generated sensory input such assound, video, images, graphics, GPS data, or other information, or mayprovide a mixed reality interface in which electronically generatedobjects are inserted in a direct or indirect view of real-worldenvironments in a manner such that they may co-exist and interact inreal time with the real-world environment and real world objects.

The sensor-enabled holster 118 may be an active (powered) or passive(non-powered) sensor that maintains and/or provides state informationregarding a weapon or other item normally disposed within the user's 102sensor-enabled holster 118. The sensor-enabled holster 118 may detect achange in state (presence to absence) and/or an action (removal)relative to the weapon normally disposed within the sensor-enabledholster 118. The detected change in state and/or action may be reportedto the portable radio 104 via its short-range transceiver. In someembodiments, the sensor-enabled holster 118 may also detect whether thefirst responder's hand is resting on the weapon even if it has not yetbeen removed from the holster and provide such information to portableradio 104. Other possibilities exist as well.

The biometric sensor wristband 120 may be an electronic device fortracking an activity of the user 102 or a health status of the user 102,and may include one or more movement sensors (such as an accelerometer,magnetometer, and/or gyroscope) that may periodically or intermittentlyprovide to the portable radio 104 indications of orientation, direction,steps, acceleration, and/or speed, and indications of health such as oneor more of a captured heart rate, a captured breathing rate, and acaptured body temperature of the user 102, perhaps accompanying otherinformation. In some embodiments, the biometric sensor wristband 120 mayinclude its own long-range transceiver and may communicate with othercommunication devices and/or with the infrastructure RAN 152 orvehicular transceiver 136 directly without passing through portableradio 104.

An accelerometer is a device that measures acceleration. Single andmulti-axis models are available to detect magnitude and direction of theacceleration as a vector quantity, and can be used to sense orientation,acceleration, vibration shock, and falling. A gyroscope is a device formeasuring or maintaining orientation, based on the principles ofconservation of angular momentum. One type of gyroscope, amicroelectromechanical system (MEMS) based gyroscope, useslithographically constructed versions of one or more of a tuning fork, avibrating wheel, or resonant solid to measure orientation. Other typesof gyroscopes could be used as well. A magnetometer is a device used tomeasure the strength and/or direction of the magnetic field in thevicinity of the device, and can be used to determine a direction inwhich a person or device is facing.

The heart rate sensor may use electrical contacts with the skin tomonitor an electrocardiography (EKG) signal of its wearer, or may useinfrared light and imaging device to optically detect a pulse rate ofits wearer, among other possibilities.

A breathing rate sensor may be integrated within the sensor wristband120 itself, or disposed separately and communicate with the sensorwristband 120 via a short range wireless or wired connection. Thebreathing rate sensor may include use of differential capacitivecircuits or capacitive transducers to measure chest displacement andthus breathing rates. In other embodiments, a breathing sensor maymonitor a periodicity of mouth and/or nose-exhaled air (e.g., using ahumidity sensor, temperature sensor, capnometer or spirometer) to detecta respiration rate. Other possibilities exist as well.

A body temperature sensor may include an electronic digital or analogsensor that measures a skin temperature using, for example, a negativetemperature coefficient (NTC) thermistor or a resistive temperaturedetector (RTD), may include an infrared thermal scanner module, and/ormay include an ingestible temperature sensor that transmits aninternally measured body temperature via a short range wirelessconnection, among other possibilities.

Although the biometric sensor wristband 120 is shown in FIG. 1 as abracelet worn around the wrist, in other examples, the biometric sensorwristband 120 may additionally and/or alternatively be worn aroundanother part of the body, or may take a different physical formincluding an earring, a finger ring, a necklace, a glove, a belt, orsome other type of wearable, ingestible, or insertable form factor.

The portable radio 104, RSM video capture device 106, laptop 114, smartglasses 116, sensor-enabled holster 118, and/or biometric sensorwristband 120 may form a personal area network (PAN) via correspondingshort-range PAN transceivers, which may be based on a Bluetooth, Zigbee,or other short-range wireless protocol having a transmission range onthe order of meters, tens of meters, or hundreds of meters.

The portable radio 104 and/or RSM video capture device 106 (or any otherdevice in FIG. 1 for that matter) may each include a locationdetermination device integrated with or separately disposed butcommunicably coupled to the portable radio 104 and/or RSM 106 and/or inrespective receivers, transmitters, or transceivers of the portableradio 104 and RSM 106 for determining a location of the portable radio104 and RSM 106. The location determination device may be, for example,a global positioning system (GPS) receiver or wireless triangulationlogic using a wireless receiver or transceiver and a plurality ofwireless signals received at the wireless receiver or transceiver fromdifferent locations, among other possibilities. The locationdetermination device may also include an orientation sensor fordetermining an orientation that the device is facing. Each orientationsensor may include a gyroscope and/or a magnetometer. Other types oforientation sensors could be used as well. The location (and/ororientation) can then be stored locally and/or transmitted via thetransmitter or transceiver to other computing devices and/or to theinfrastructure RAN 152.

The vehicle 132 may include the vehicular computing device 133, thevehicular video camera 134 and/or microphone 135, the vehiculartransceiver 136, and the head and/or eye-tracking device 139, all ofwhich may be coupled to one another via a wired and/or wireless vehiclearea network (VAN), perhaps along with other sensors physically orcommunicatively coupled to the vehicle 132. The vehicular transceiver136 may include a long-range transceiver for directly wirelesslycommunicating with mobile computing devices such as the portable radio104, the RSM 106, and the laptop 114 via wireless link(s) 142 and/or forwirelessly communicating with the infrastructure RAN 152 via wirelesslink(s) 144. The vehicular transceiver 136 may further include ashort-range wireless transceiver or wired transceiver for communicablycoupling between the vehicular computing device 133 and/or the vehicularvideo camera 134 in the VAN. The vehicular computing device 133 may, insome embodiments, include the vehicular transceiver 136 and/or thevehicular video camera 134 integrated therewith, and may operate tostore and/or process video and/or audio produced by the video camera 134and/or transmit the captured video and/or audio as a video and/or audiostream to the portable radio 104, other mobile computing devices, and/orthe infrastructure RAN 152 for further analysis. The omni-directional orunidirectional microphone 135, or an array thereof, may be integrated inthe video camera 134 and/or at the vehicular computing device 133 (oradditionally or alternatively made available at a separate location ofthe vehicle 132) and communicably coupled to the vehicular computingdevice 133 and/or vehicular transceiver 136 for capturing audio andstoring, processing, and/or transmitting the audio in a same or similarmanner as set forth above with respect to the RSM 106.

The vehicle 132 may be a human-operable vehicle, or may be aself-driving vehicle operable under control of vehicular computingdevice 133 perhaps in cooperation with video camera 134 (which mayinclude a visible-light camera, an infrared camera, a time-of-flightdepth camera, and/or a light detection and ranging (LiDAR) device).Command information and/or status information such as location and speedmay be exchanged with the self-driving vehicle via the VAN and/or thePAN (when the PAN is in range of the VAN or via the VAN's infrastructureRAN link).

The vehicle 132 and/or transceiver 136, similar to the portable radio104 and/or respective receivers, transmitters, or transceivers thereof,may include a location (and/or orientation) determination deviceintegrated with or separately disposed in the vehicular computing device133 and/or transceiver 136 for determining (and storing and/ortransmitting) a location (and/or orientation) of the vehicle 132.

In some embodiments, instead of a vehicle 132, a land, air, orwater-based drone with same or similar audio and/or video andcommunications capabilities and same or similar self-navigatingcapabilities as set forth above may be disposed, and may similarlycommunicate with the user's 102 PAN and/or with the infrastructure RAN152 to support the user 102 in the field.

The VAN may communicatively couple with the PAN disclosed above when theVAN and the PAN come within wireless transmission range of one another,perhaps after an authentication takes place there between, and one ofthe VAN and the PAN may provide infrastructure communications to theother, depending on the situation and the types of devices in the VANand/or PAN and may provide interoperability and communication linksbetween devices (such as video cameras) and sensors within the VAN andPAN.

Although the RSM 106, the laptop 114, the smart glasses 116, and thevehicle 132 are illustrated in FIG. 1 as providing example video camerasand/or microphones for use in capturing audio and/or video streams,other types of cameras and/or microphones could be used as well,including but not limited to, fixed or pivotable video cameras securedto lamp posts, automated teller machine (ATM) video cameras, other typesof body worn cameras such as head-mounted cameras, other types ofvehicular cameras such as roof-mounted cameras, or other types of audioand/or video recording devices accessible via a wired or wirelessnetwork interface same or similar to that disclosed herein.

Infrastructure RAN 152 is a radio access network that provides for radiocommunication links to be arranged within the network between aplurality of user terminals. Such user terminals may be mobile and maybe known as ‘mobile stations’ or ‘mobile devices,’ and may include anyone or more of the electronic computing devices illustrated in FIG. 1,among other possibilities. At least one other terminal, e.g. used inconjunction with mobile devices, may be a fixed terminal, e.g. a basestation, eNodeB, repeater, and/or access point. Such a RAN typicallyincludes a system infrastructure that generally includes a network ofvarious fixed terminals, which are in direct radio communication withthe mobile devices. Each of the fixed terminals operating in the RAN mayhave one or more transceivers which may, for example, serve mobiledevices in a given region or area, known as a ‘cell’ or ‘site’, by radiofrequency (RF) communication. The mobile devices that are in directcommunication with a particular fixed terminal are said to be served bythe fixed terminal. In one example, all radio communications to and fromeach mobile device within the RAN are made via respective serving fixedterminals. Sites of neighboring fixed terminals may be offset from oneanother and may provide corresponding non-overlapping or partially orfully overlapping RF coverage areas.

Infrastructure RAN 152 may operate according to an industry standardwireless access technology such as, for example, an LTE, LTE-Advance, or5G technology over which an OMA-PoC, a VoIP, an LTE Direct or LTE Deviceto Device, or a PoIP application may be implemented. Additionally oralternatively, infrastructure RAN 152 may implement a WLAN technologysuch as Wi-Fi perhaps operating in accordance with an IEEE 802.11standard (e.g., 802.11a, 802.11b, 802.11g) or such as a WiMAX perhapsoperating in accordance with an IEEE 802.16 standard.

Infrastructure RAN 152 may additionally or alternatively operateaccording to an industry standard LMR wireless access technology suchas, for example, the P25 standard defined by the APCO, the TETRAstandard defined by the ETSI, the dPMR standard also defined by theETSI, or the DMR standard also defined by the ETSI. Because thesesystems generally provide lower throughput than the broadband systems,they are sometimes designated as narrowband RANs.

Communications in accordance with any one or more of these protocols orstandards, or other protocols or standards, may take place over physicalchannels in accordance with one or more of a TDMA (time divisionmultiple access), FDMA (frequency divisional multiple access), OFDMA(orthogonal frequency division multiplexing access), or CDMA (codedivision multiple access) technique.

OMA-PoC, in particular and as one example of an infrastructure broadbandwireless application, enables familiar PTT and “instant on” features oftraditional half duplex mobile devices, but uses mobile devicesoperating over modern broadband telecommunications networks. UsingOMA-PoC, wireless mobile devices such as mobile telephones and notebookcomputers can function as PTT half-duplex mobile devices fortransmitting and receiving. Other types of PTT models and multimediacall models (MMCMs) could be used as well.

Floor control in an OMA-PoC session is generally maintained by a PTTserver that controls communications between two or more wireless mobiledevices. When a user of one of the mobile devices keys a PTT button, arequest for permission to speak in the OMA-PoC session is transmittedfrom the user's mobile device to the PTT server using, for example, areal-time transport protocol (RTP) message. If no other users arecurrently speaking in the PoC session, an acceptance message istransmitted back to the user's mobile device and the user can then speakinto a microphone of the device. Using standardcompression/decompression (codec) techniques, the user's voice isdigitized and transmitted using discrete auditory data packets (e.g.,together which form an auditory data stream over time), such asaccording to RTP and internet protocols (IP), to the PTT server. The PTTserver then transmits the auditory data packets to other users of thePoC session (e.g., to other mobile devices in the group of mobiledevices or talkgroup to which the user is subscribed), using forexample, one or more of a unicast, point to multipoint, or broadcastcommunication technique.

Infrastructure narrowband LMR wireless systems, on the other hand, mayoperate in either a conventional or trunked configuration. In eitherconfiguration, a plurality of mobile devices is partitioned intoseparate groups of mobile devices.

In a conventional narrowband radio system, each mobile device in a groupis selected to a particular radio channel (frequency or frequency & timeslot) for communications associated with that mobile device's group.Thus, each group is served by one channel, and multiple groups may sharethe same single frequency (in which case, in some embodiments, group IDsmay be present in the group data to distinguish between groups using thesame shared frequency).

In contrast, a trunked narrowband radio system and its mobile devicesuse a pool of traffic channels for virtually an unlimited number ofgroups of mobile devices (e.g., talkgroups). Thus, all groups are servedby all channels. The trunked radio system works to take advantage of theprobability that not all groups need a traffic channel for communicationat the same time. When a member of a group requests a call on a controlor rest channel on which all of the mobile devices at a site idleawaiting new call notifications, in one embodiment, a call controllerassigns a separate traffic channel for the requested group call, and allgroup members move from the assigned control or rest channel to theassigned traffic channel for the group call. In another embodiment, whena member of a group requests a call on a control or rest channel, thecall controller may convert the control or rest channel on which themobile devices were idling to a traffic channel for the call, andinstruct all mobile devices that are not participating in the new callto move to a newly assigned control or rest channel selected from thepool of available channels. With a given number of channels, a muchgreater number of groups can be accommodated in a trunked radio systemas compared with a conventional radio system.

Group calls may be made between wireless and/or wireline participants inaccordance with either a narrowband or a broadband protocol or standard.Group members for group calls may be statically or dynamically defined.That is, in a first example, a user or administrator working on behalfof the user may indicate to the switching and/or radio network (perhapsat a call controller, PTT server, zone controller, or mobile managemententity (MME), base station controller (BSC), mobile switching center(MSC), site controller, Push-to-Talk controller, or other networkdevice) a list of participants of a group at the time of the call or inadvance of the call. The group members (e.g., mobile devices) could beprovisioned in the network by the user or an agent, and then providedsome form of group identity or identifier, for example. Then, at afuture time, an originating user in a group may cause some signaling tobe transmitted indicating that he or she wishes to establish acommunication session (e.g., group call) with each of the pre-designatedparticipants in the defined group. In another example, mobile devicesmay dynamically affiliate with a group (and also disassociate with thegroup) perhaps based on user input, and the switching and/or radionetwork may track group membership and route new group calls accordingto the current group membership.

In some instances, broadband and narrowband systems may be interfacedvia a middle-ware system that translates between a narrowband PTTstandard protocol (such as P25) and a broadband PTT standard protocol(such as OMA-PoC). Such intermediate middle-ware may include amiddleware server for performing the translations and may be disposed inthe cloud, disposed in a dedicated on-premises location for a clientwishing to use both technologies, or disposed at a public carriersupporting one or both technologies. For example, and with respect toFIG. 1, such a middle-ware server may be disposed in infrastructure RAN152 at controller 156 or at a separate cloud computing cluster 162communicably coupled to controller 156 via internet protocol (IP)network 160, among other possibilities.

The infrastructure RAN 152 is illustrated in FIG. 1 as providingcoverage for the portable radio 104, RSM video capture device 106,laptop 114, smart glasses 116, and/or vehicle transceiver 136 via asingle fixed terminal 154 coupled to a single controller 156 (e.g.,radio controller, call controller, PTT server, zone controller, MME,BSC, MSC, site controller, Push-to-Talk controller, or other networkdevice) and including a dispatch console 158 operated by a dispatcher.In other embodiments, additional fixed terminals and additionalcontrollers may be disposed to support a larger geographic footprintand/or a larger number of mobile devices.

The controller 156 illustrated in FIG. 1, or some other backendelectronic computing device existing on-premises or in the remote cloudcompute cluster 162 accessible via the IP network 160 (such as theInternet), may additionally or alternatively operate as a back-endelectronic digital assistant, a back-end audio and/or video processingelectronic computing device, and/or a remote cloud-based storage deviceconsistent with the remainder of this disclosure.

The IP network 160 may comprise one or more routers, switches, LANs,WLANs, WANs, access points, or other network infrastructure, includingbut not limited to, the public Internet. The cloud compute cluster 162may be comprised of a plurality of computing devices, such as the oneset forth in FIG. 2, one or more of which may be executing none, all, ora portion of an electronic digital assistant service, sequentially or inparallel, across the plurality of computing devices. The plurality ofcomputing devices comprising the cloud compute cluster 162 may begeographically co-located or may be separated by inches, meters, ormiles, and inter-connected via electronic and/or optical interconnects.Although not shown in FIG. 1, one or more proxy servers or loadbalancing servers may control which one or more computing devicesperform any part or all of the electronic digital assistant function.

System 100 may additionally include a physical street sign 170, such asan intersection sign, that includes alphanumeric text and/or images thatmay identify, for example, two cross streets meeting one another at ornear the location of the street sign 170. In other embodiments, variousdifferent types of signs, including physical or dynamically updateableelectronic signs indicating construction updates, detours, events,traffic updates, advertisements, logos, building addresses, or otherinformation may be implemented as well. The license plate 172 may be aphysical or electronic display attached to the vehicle 132 that includesa unique identifier to uniquely identify (e.g., within a local region orarea, county, city, state, or country) the vehicle 132 and may be linkedto other information such as an owner, driver, employee, licensee,company, insurance information, traffic infraction information, makeand/or model of the vehicle, or other information associated with thevehicle. The street sign 170 and the license plate 172 are merelyexample alphanumeric elements on graphical objects that will bedescribed in more detail below with respect to the process 300 of FIG.3.

Finally, although FIG. 1 describes a communication system 100 generallyas a public safety communication system including a user 102 generallydescribed as a police officer and vehicle 132 generally described as apolice cruiser, in other embodiments, the communications system 100 mayadditionally or alternatively be a retail communications systemincluding a user 102 that may be an employee of a retailer and a vehicle132 that may be a vehicle for use by the user 102 in furtherance of theemployee's retail duties (e.g., a shuttle or self-balancing scooter). Inother embodiments, the communications system 100 may additionally oralternatively be a warehouse communications system including a user 102that may be an employee of a warehouse and a vehicle 132 that may be avehicle for use by the user 102 in furtherance of the employee's retailduties (e.g., a forklift).

In still further embodiments, the communications system 100 mayadditionally or alternatively be a private security communicationssystem including a user 102 that may be an employee of a privatesecurity company and a vehicle 132 that may be a vehicle for use by theuser 102 in furtherance of the private security employee's duties (e.g.,a private security vehicle or motorcycle). In even further embodiments,the communications system 100 may additionally or alternatively be amedical communications system including a user 102 that may be a doctoror nurse of a hospital and a vehicle 132 that may be a vehicle for useby the user 102 in furtherance of the doctor or nurse's duties (e.g., amedical gurney or ambulance). In a last example embodiment, thecommunications system 100 may additionally or alternatively be a heavymachinery communications system including a user 102 that may be aminer, driller, or extractor at a mine, oil field, or precious metal orgem field and a vehicle 132 that may be a vehicle for use by the user102 in furtherance of the miner, driller, or extractor's duties (e.g.,an excavator, bulldozer, crane, or front loader). Other possibilitiesexist as well.

b. Device Structure

Referring to FIG. 2, a schematic diagram illustrates an electroniccomputing device 200 for operating an electronic digital assistantaccording to some embodiments of the present disclosure. Electroniccomputing device 200 may be, for example, embodied in the portable radio104, RSM video capture device 106, laptop 114, smart glasses 116,vehicular computing device 133, controller 156, or some other electroniccomputing device not illustrated in FIG. 1 including the remote cloudcompute cluster 162 described above, and/or may be a distributedcomputing device across two or more of the foregoing (or multiple of asame type of one of the foregoing) and linked via wired and/or wirelesscommunication link(s). As shown in FIG. 2, computing device 200 includesa communications unit 202 coupled to a common data and address bus 217of a processing unit 203. The computing device 200 may also include aninput unit (e.g., keypad, pointing device, touch-sensitive surface,etc.) 206 and an electronic display screen 205, each coupled to be incommunication with the processing unit 203.

A microphone 220 may be present for capturing audio from a user and/orother environmental or background audio that is further processed byprocessing unit 203 in accordance with the remainder of this disclosureand/or is transmitted as voice or audio stream data, or as acousticalenvironment indications, by communication unit 202 to other portableradios and/or other electronic computing devices. An imaging device 221may provide video (still or moving images) of an area in a field of viewof the computing device 200 (and perhaps also matching or containing amatching a field of view of a user associated therewith) for furtherprocessing by the processing unit 203 and/or for further transmission bycommunications unit 202. A communications speaker 222 may be present forreproducing audio that is decoded from voice or audio streams of callsreceived via the communication unit 202 from other portable radios, fromdigital audio stored at the computing device 200, from other ad-hoc ordirect mode devices, and/or from an infrastructure RAN device, or mayplay back alert tones or other types of pre-recorded audio.

The processing unit 203 may include a code Read Only Memory (ROM) 212coupled to the common data and address bus 217 for storing data forinitializing system components. The processing unit 203 may furtherinclude a microprocessor 213 coupled, by the common data and address bus217, to a Random Access Memory (RAM) 204 and a static memory 216.

The communications unit 202 may include one or more wired and/orwireless input/output (I/O) interfaces 209 that are configurable tocommunicate with other devices, such as a portable radio, laptop,wireless RAN, and/or vehicular transceiver.

For example, the communications unit 202 may include one or morewireless transceivers 208, such as a DMR transceiver, a P25 transceiver,a Bluetooth transceiver, a Wi-Fi transceiver perhaps operating inaccordance with an IEEE 802.11 standard (e.g., 802.11a, 802.11b,802.11g), an LTE transceiver, a WiMAX transceiver perhaps operating inaccordance with an IEEE 802.16 standard, and/or other similar type ofwireless transceiver configurable to communicate via a wireless radionetwork.

The communications unit 202 may additionally or alternatively includeone or more wireline transceivers 208, such as an Ethernet transceiver,a USB transceiver, or similar transceiver configurable to communicatevia a twisted pair wire, a coaxial cable, a fiber-optic link, or asimilar physical connection to a wireline network. The transceiver 208is also coupled to a combined modulator/demodulator 210.

The microprocessor 213 has ports for coupling to the input unit 206 andthe microphone unit 220, and to the display screen 205, imaging device221, and speaker 222. Static memory 216 may store operating code 225 forthe microprocessor 213 that, when executed, performs one or more of thecomputing device steps set forth in FIG. 3 and accompanying text. Staticmemory 216 may also store, permanently or temporarily, cached imagesand/or video frames (full or partial) of captured objects and/ortranslated identifications and/or contents of objects detected incaptured image or video frames, including but not limited to extractedalphanumeric text, in accordance with the description below.

Static memory 216 may comprise, for example, a hard-disk drive (HDD), anoptical disk drive such as a compact disk (CD) drive or digitalversatile disk (DVD) drive, a solid state drive (SSD), a tape drive, aflash memory drive, or a tape drive, to name a few.

2. Processes for Processing Video Relative to a Determined User'sPerception of Such Video and Tailoring an Electronic Digital Assistant'sInquiry Response to the User's Subsequent Query.

Turning now to FIG. 3, a flowchart diagram illustrates a process 300 foran electronic computing device operating as an electronic digitalassistant to computationally process video relative to a determineduser's perception of such video and to tailor an electronic digitalassistant's inquiry response to a subsequent query from the useraccordingly. While a particular order of processing steps, messagereceptions, and/or message transmissions is indicated in FIG. 3 forexemplary purposes, timing and ordering of such steps, receptions, andtransmissions may vary where appropriate without negating the purposeand advantages of the examples set forth in detail throughout theremainder of this disclosure. The computing device may execute process300 at power-on, at some predetermined periodic time period thereafter,in response to a trigger raised locally at the device via an internalprocess or via an input interface (e.g., the user enabling a particularfeature associated with process 300 via an input interface or thecomputing device detecting that the computing device has entered aparticular area or vehicle or that a user thereof has exited aparticular area or vehicle, among other possibilities), or in responseto detecting a trigger (including receipt of media content forprocessing in accordance with process 300) from some other portableradio, vehicle, infrastructure controller, or other computing device towhich it is communicably coupled, among other possibilities.

The computing device executing process 300 may include an edge devicesame or similar to any one or more of the portable radio 104, the RSM106, the laptop 114, the smart glasses 116, or the vehicle computingdevice 133 illustrated in FIG. 1, may include an infrastructure devicesame or similar to the controller 156 of FIG. 1, may include some otherin-field, infrastructure RAN, or remote cloud computing cluster 162device, or may include two or more of the foregoing operating in adistributed computing manner, among other possibilities.

Process 300 begins at step 302 where an electronic computing deviceoperating as an electronic digital assistant receives, from a videocapture device configured to track a gaze direction of a first user, avideo stream including a first field-of-view substantially matching afield-of-view of the first user. The video capture device may be, forexample, a pair of smart glasses such as smart glasses 116 of FIG. 1that automatically physically track a field-of-view of the user 102wearing them due to their physical design and positioning on the user'sbody and thus produces a video stream substantially directly matching(e.g., plus or minus 5° in any one direction) a field of view of itswearer. Additionally or alternatively, the smart glasses 116 may includesome form of eye-tracking mechanism that may provide a more fine-grainedinput as to what portion of an overall vertical and horizontalfield-of-view captured by a video capture device integrated with thesmart glasses 116 that the user's 102 eyes are directed at or focusedon. Of course, and as set forth earlier, in other instances, a separatehead and/or eye-tracking electronic device such as the head and/oreye-tracking device 139 of FIG. 1 may be detached from the user 102 itis tracking but may similarly generate and provide information to thecomputing device (such as vehicular computing device 133) regarding asub-portion of the user's 102 field-of-view captured via a vehicularcamera such as vehicular video camera 134 that the user's 102 headand/or eyes are directed at or focused on. As just one example, the headand/or eye-tracking device may determine that, of an overall 120°field-of-view captured by an associated video capture device, a user'shead and/or eyes are focused on an upper-left quadrant of the capturedfield-of-view video representing 60° in the vertical direction and 60°in the horizontal direction (e.g., ¼ of the originally capturedfield-of-view). Of course, other ranges of captured field-of-view may becaptured by the associated video capture device (such as between 60° and360°) and other fractional sub-portions of the captured field-of-viewmay be identified by the head and/or eye tracking device (such as 1/10to ½ of the captured field-of-view perhaps depending on whether one orboth of head-tracking and eye-tracking devices are present and providingfield-of-view limiting indications to the electronic computing device).

Returning to the example driver's head and/or eye-tracking device 139example of FIG. 1, the device 139 may be a rear-facing (in relation tothe rear of the car) optical recording device that is capable oftracking a location of the driver's (e.g., user's 102) head and/or eyegaze and determining, based on the optical tracking, a direction inwhich the user 102 is looking (where 0° horizontally is directly forwardand 90° is tangentially to the user's 102 right and which may continuearound 360° to the 0° horizontal position again, and where 0° verticallyis flush with a detected horizon and 90° is tangentially above the user102 and which may continue around 360° to the 0° vertical positionagain). For example, the rear-facing device head and/or eye-trackingdevice 139 may optically track the user's 102 gaze using infrared lightreflections to track movements in a center of the user's 102 pupil,front of the cornea, and/or back of the lens, or by tracking movementsin detected retinal blood vessels. In other embodiments, the device 139may use face-detection on captured 2D images to detect a direction inwhich the user's 102 face is directed. Still further, a depth camera maybe used to capture 3D depth images and to detect a direction in whichthe user's 102 face is directed.

In still other embodiments, the device 139 may be an electro-mechanicalsystem that may be physically coupled to the user's 102 head and mayelectro-mechanically detect the user's 102 head direction and transmitphysical direction information in a manner similar to the foregoing.Still further, the device 139 may be an electromagnetic system disposedto detect changes in magnetic fields as the user 102 moves his or herhead, and may similarly electronically communicate the detected user's102 head position to the electronic computing device in accordance withthe foregoing. Other possibilities for electro-mechanical tracking existas well. And still other methods of tracking a user's 102 head and/orgaze direction are possible as well, inside of or outside of the vehicle132 as described above. Other possibilities for tracking exist as well,including a combination of two or more of the foregoing.

The video stream received at the electronic computing device at step 302may include a full field-of-view (e.g., larger than the user's field ofview, such as from a 360° video capture device) captured by the videocapture device, which may then be sub-sampled in accordance withadditional head and/or eye-tracking information separately received fromthe video capture device (e.g., smart glasses) or some other separatehead and/or eye-tracking device, or the video stream received at theelectronic computing device at step 302 may be provided in an alreadysub-sampled state, wherein the video capture device or some othercomputing device communicably coupled to the video capture device (andperhaps embedded or incorporated in the head and/or eye tracking device)may perform the sub-sampling prior to providing the video stream to theelectronic computing device at step 302.

The manner of extracting a limited field-of-view from a broaderfield-of-view included in the video stream, wherever performed, may varydepending upon the underlying media encoding method of the video stream.For media formats that fully encode all image data for each frame,sub-selecting the limited field-of-view may simply involve extractingthose quadrants of the video stream corresponding to the more limitedfield-of view using the field-of-view information set forth above. Formedia formats that encode only changes in image data between key frames,sub-selecting the limited field-of-view may involve extracting thosequadrants (or other region definitions including but not limited topoint and radius, polygonal definitions, etc.) of the key framescorresponding to the extracted and limited fields-of-view, andre-encoding changes between the extracted key frame portions into a newencoded video stream using the field-of-view information set forthabove. Other methods of sub-selecting in field-of-view are possible aswell. For example, saccades are rapid movements between fixation points.These movements have unique patterns and are detectable. These movementscan be used to mark content the user probably absorbed, at the fixationpoints, and content in areas between those points are not marked asabsorbed by the user.

In some embodiments, the video capture device providing the video streamat step 302 may include or be associated with a separate audio capturedevice, and an audio stream may be provided to the electronic computingdevice at step 302 accompanying the video stream for further use at step304. For example, the microphone 135 coupled to the video camera 134 maycapture ambient audio, including voice audio generated by user 102, andprovide a corresponding audio stream to vehicular computing device 133,controller 156, and/or cloud computer cluster 162, among otherpossibilities.

After receiving the video stream at step 302 (and optionally acorresponding audio stream as well), process 300 continues to step 304,where the electronic computing device identifies an object in the videostream first field-of-view remaining in the first field-of-view for adetermined threshold period of time and, in response, processes theobject via a video processing algorithm. The video processing algorithmgenerates object information that is provided to the electroniccomputing device, which may then store the object information and anyother corresponding metadata (including audio) in a non-volatile storefor future reference in responding to queries from the user. Inembodiments where the electronic computing device is provided a fullfield-of-view video stream and field-of-view identification informationidentifying a subset of the field-of-view of a user associated with thevideo stream, the electronic computing device may separately processportions of the video stream matching the field-of-view identificationinformation and store object information identified in the matchingfield-of-view portions of the video stream as in-view object informationand portions of the video stream not matching the field-of-viewidentification information and store object information identified asoutside of the matching field-of-view portions of the video stream asout-of-view object information.

The identified object may be any type of material thing that can be seenand (hypothetically though not literally) touched by the user, such as aperson, building, tool, animal, sign, event, vehicle, or other type ofobject. The electronic computing device may apply an object-recognitionalgorithm having access to a database of known objects to the receivedvideo stream (including the in-field-of-view stream and/or theout-of-field-of-view video stream) and identify one or more objects thatmatch known object(s) from the object recognition database. Variousimage processing algorithms may be used to match objects in the videostream received at step 302 to a known object, including but not limitedto geometric hashing, edge detection, scale-invariant feature transform(SIFT), speeded-up robust features (SURF), neural networks, deeplearning, genetic, gradient-based and derivative-based matchingapproaches, Viola-Jones algorithm, template matching, or imagesegmentation and blob analysis. Other possibilities exist as well. Oncean object is matched, metadata describing the object from the objectrecognition database (e.g., identifying the object as a person, avehicle, a building, a tool, an animal, a sign, an event, a vehicle, orsome other type of object) may be embedded and stored within the videostream prior to storing the video stream, or may be stored in a separatevideo stream metadata file and stored accompanying the video stream (andboth associated with the user). Additional metadata regarding theidentified object may be identified and/or extracted from the videostream or other sources (such as audio, as set forth in more detailbelow) and stored as well, including but not limited to a color of theobject, a time (point in time or a time window in which it is partially(25% or less) or mostly (50%-90%) or entirely (100%) visible) duringwhich the object appeared in the video stream, a location at which theobject was detected (perhaps received via a location-determinationdevice integrated with the electronic computing device or communicablycoupled to the electronic computing device, or determined via videoanalytics of surrounding objects such as street signs), a role and/oridentity of the user, a current incident with which the user iscurrently associated, or some other contextual information.

In one example, the object may be an alphanumeric text object (such as apolice report) or some graphical object including alphanumeric text(such as the street sign 170 of FIG. 1), and stored object informationassociated with such an object may include an identity of the object(e.g., “police report” or “street sign” or “license plate” as determinedvia the object recognition algorithm and database) and an opticalcharacter recognition translation of the alphanumeric contents of theobject (e.g., a name, location, address, time, witness name(s), incidenttitle or identifier, or status with respect to alphanumeric content ofthe police report or respective street names appearing on the streetsign graphical object or a respective license plate number appearing ona license plate graphical object) perhaps accompanying the additionalmetadata already noted above. Other types of objects containingalphanumeric content could be similarly processed by the videoprocessing algorithm and similar object information generated and storedaccompanying the video stream and associated with the user.

In other examples, the object may be a graphical object that does ordoes not otherwise include alphanumeric text, and the object informationmay include an identification of a type of the graphical object (e.g., atea pot), a definition associated with the identified type of graphicalobject (e.g., a vessel for boiling water), and/or a unique identity ofthe graphical object (e.g., tea pot at suspect John Smith's residence)perhaps accompanying the additional metadata already noted above.

In still further examples, the object may be a capture of a human face,and the electronic computing device may apply a face-recognitionalgorithm having access to a database of known faces to the video streamand the stored object information may include a graphical capture of theface, unique characteristics of the identified human face graphicalobject (e.g., such as distance measurements between facial features,measured sizes of facial features, etc.), and/or a determined uniqueidentity of the person determined to match the human face graphicalobject via a face-recognition algorithm and database function. Variousfacial recognition image processing algorithms may be used to matchstored faces associated with known criminal organization associates,including but not limited to a high dimensional local binary pattern(LBP) algorithm, a Fisher vector algorithm, a Joint Bayesian algorithm,and an associated predict algorithm. Other types of facial recognitionimage processing algorithms could be used as well.

In an additional embodiment, the electronic computing device may furtheraccess audio portions of the audio stream (if available) matching thevideo stream in order to extract and capture additional objectinformation relative to objects identified in the video stream. Forexample, the user associated with the video stream may provide auditorycomments contemporaneously with the capture and generation of the videostream that may provide additional description and/or aid the electroniccomputing device in identifying one or more objects in the video stream.The user may comment that “the tea kettle is still hot”, which may thenbe captured and used by the object recognition algorithm to aid inidentifying the object as a tea kettle, confirm the otherwiseidentification of the object as a tea kettle, and/or provide additionalmetadata (e.g., it was “hot”) for storage as objection informationaccompanying the identity of the object associated with the user. Suchcaptured audio can also aid the electronic computing device indetermining which portions of a police report within the user's field ofview the user has read and comprehended in those instances where theuser may read portions of the police report aloud, and/or may aid theoptical character recognition algorithm in converting (and/or confirminga transcription of) the text in the report to text for storage in thestored object information. Other possibilities exist as well.

The electronic computing device may subsequently actively maintain oneor both sets of in-view object information and out-of-view objectinformation associated with the user in accordance with one or moreobject information maintenance rule sets. The object informationmaintenance rule sets may be applied to all users, may vary based on arole or identity of the user, or may be individually set by the user,among other possibilities. One rule in the object informationmaintenance rule set may set forth an expiration timer that identifieshow long object information identified and stored at step 304 ismaintained as associated with the user. After expiration of the timer,the object information may be archived, deleted, removed, and/orotherwise made inaccessible for subsequent steps of process 300,including step 312 that generates a response to a user inquiry as afunction of the object information associated with the user. A thresholdperiod of time for the expiration timer is a predetermined estimatedtime that the user will independently retain information relative to thein-view object information after viewing the object in his or her fieldof view. The threshold period of time may be a statically appliedthreshold period of time in the range of four to twenty-four hours, orten to sixteen hours, or twelve hours. Other static periods of time arepossible as well. In other embodiments, the applied threshold period oftime for the expiration timer may be dynamic and set based on one ormore context parameters, including but not limited to one or more of adetermined age of the user associated with the video stream (asretrieved from an employment database, for example, the higher the agethe lower the threshold period of time), a measured amount of time theobject remained in the field of view of the user (the longer the objectremained in the user's field of view, the higher the threshold period oftime), a number of times the user looked away from (i.e., the objectleft the user's field of view) and then returned to the object (i.e.,the object returned to the user's field of view, the higher number oftimes, the higher the threshold period of time), a size of the object,certain key words appearing on the object, or some other contextparameter relating to the user or to the object. In other embodiments, atime passed since an object was last seen may be mapped to a probabilityof recollection, and a probability of recollection value calculated as afunction of the time passed (amongst other parameters including thoseset forth above) and compared to a predetermined organization-wide orper-user minimum recollection probability value to determine if theelectronic computing device should continue to maintain an associationbetween the object information identified and stored at step 304 and theuser.

In still further embodiments, some objects may be identified as exemptfrom the expiration time or subject to a lower minimum recollectionprobability value (including none at all). These may be items, forexample, that have special characteristics determined that an averageuser (or a particular user) would normally maintain for a long orindefinite period of time. For example, a location or date, or objectsinvolved in, a traumatic event such as a detected explosion in a fieldof view of the user may be considered as exempt from the expiration timeor subject to a lower minimum recollection probability value.

Process 300 then continues to step 308, where the electronic processingdevice subsequently receives an inquiry for information from the userassociated with the video stream. The inquiry may, for example, be inthe form of a voice request captured via an integrated microphone at theelectronic computing device, or a voice request captured at some otherdevice and provided to the electronic computing device (e.g., in anaudio or transcribed form) via one or more wired or wireless networksconsistent with the system architecture described above with respect tothe system 100 of FIG. 1. In other embodiments, the inquiry may be inthe form of a text message, e-mail, instant message, or some other inputentry mechanism.

The inquiry may include one or more of a time indication, a locationindication, an incident indication, a person indication, an objectindication, or some other indication which may or may not match one ormore object information generated and stored at step 304.

As one example, the content of the inquiry may request “what were thenames of the cross streets at the incident in the Bucktown neighborhoodof Chicago?” The request may be a voice request made to the electroniccomputing device by the user (an officer) after the officer returnedfrom the incident to a police station and began to write up a report. Asanother example, the content of the inquiry may request “what was thelicense plate on the green car at the incident scene this morning?” or“what was the name of the first witness in the incident report for theretail store robbery?” Different inquiries may be made in differentindustries outside of public safety as well. For example, a retailsecurity employee may request “what was the name of the product thesuspect thief took off the shelf and put in his jacket?”, and/or anoil/gas engineer may ask “what was the identity of the leaking well Iran across at around 3 pm yesterday?”

Process 300 then continues to step 310, where the electronic processingdevice determines that the user inquiry (e.g., also herein identified asa user search query) is related to previously stored object information.For example, the electronic computing device may voice-to-text convertthe inquiry (if not already in text form), parse the inquiry, andattempt to match one or more search terms in the inquiry to previouslystored object information associated with the user.

Additionally or alternatively, time and/or location indications in thequery may be matched against stored object information includingcorresponding time and/or location indications.

As one particular example, a user search query of “what was the licenseplate on the green car at the incident scene this morning?” may beparsed and matched by the electronic computing against stored objectinformation that includes a date stamp of the same day, a timeindication occurring in the morning (e.g., 12:01 AM-11:59 AM), taggedwith an incident identifier associated with a car accident, and/or anobject descriptor including the terms “green” and “car”.

As another particular example, a user search query of “what was thewitnesses' name from the police report about the WAL-STORE robbery Iread yesterday?” may be matched by the electronic computing againststored object information that includes a date stamp of the prior day,tagged with an incident identifier associated with a theft, and/or anobject descriptor including the terms “police report” and/or“WAL-STORE.”

As a still further example, a search query of “what was the gas stationemployee John's last name that I spoke with yesterday?” may be matchedby the electronic computing against stored object information thatincludes a date stamp of the prior day, tagged with a person's nameincluding John as a first name (e.g., determined via facial recognition,via scan of a name badge, or via audio input detection), and/or anobject descriptor or location metadata including the term “gas station.”

Process 300 then continues to step 312, where the electronic processingdevice provides a response to the user inquiry as a function of thepreviously stored object information associated with the user. Theresponse to the user inquiry could be provided in a number of ways. Forexample, the response could be in the form of a voice output that isre-produced locally at the electronic computing device, or transmittedto some other mobile or electronic computing device associated with theuser for reproduction. For example, providing the response may includegenerating a text-based output for reproduction at the electroniccomputing device or some other mobile or vehicular computing deviceassociated with the user (e.g., including an output transducer speakerassociated with the user) via a text-to-voice software component at theelectronic computing device, mobile device, or vehicular computingdevice that converts the text to voice for output to the user via thelocal output transducer speaker. For example, an electronic digitalassistant operating at controller 156 or RSM 106 may provide theresponse by generating an auditory output text file and transmitting itto laptop 114 for reproduction. As another example, an electronicdigital assistant operating at laptop 114 may provide the response bygenerating an auditory output text file for local reproduction at thelaptop 114.

Alternatively, providing the response at step 312 may include generatinga digital audio file with digitally-encoded speech that recites asubstantive content generated as a function of the object information atstep 312 for reproduction to the user. For example, an electronicdigital assistant operating at controller 156 or RSM 106 may generate anauditory output digitally-encoded speech file and transmit it to laptop114 for reproduction. As another example, an electronic digitalassistant operating at laptop 114 may generate an auditory outputdigitally-encoded speech file for local reproduction at the laptop 114.

Of course, in other embodiments, providing the response may merelyinclude transmitting an instant message, a text message, an e-mail, orsome other text-based query response that provides an answer to the userquery as a function of the object information. Other possibilities existas well.

In still other embodiments, the response may be provided to the user (orto a group including the user) by playing back audio of the response(generated in any of the ways set forth above) over an established PTTnarrowband or broadband voice channel (private call or group callsession), using any one or more of the broadband and narrowbandapplications or protocols set forth earlier.

In a particular example where the related object information identifiedat step 310 was one of an alphanumerical text object and a graphicalobject including alphanumerical text, the related object information mayinclude alphanumerical text corresponding to the alphanumerical textobject or extracted from the graphical object, and the response to theuser inquiry at step 312 may include at least a portion of, or all of,one of the alphanumerical text itself or a transformation of thealphanumerical text into an audio reproduction. Using the exampleinquiry set forth earlier, a user search query of “what was the licenseplate on the green car at the incident scene this morning?” may bematched against corresponding related object information at step 310 andthe electronic computing device may identify a license platealphanumeric content matching the user query of “XYZ123 IL” and providea response at step 312 of an auditory recitation of the alphanumericcontext “XYZ123 IL” in some manner as already described above, or mayprovide the alphanumeric text back to the requesting user in any numberof other manners, including a text message, e-mail, or instant message,among other possibilities. A similar response could be provided wherethe inquiry is for a street sign with the response includingalphanumeric text extracted from a street-side physical or electronicsign.

In some embodiments wherein the alphanumerical text may not be fullyextracted from the graphical object, an indication may be stored in theobject information noting the existence of additional potential textthat could not be translated with a high level of confidence (e.g.,greater than 50% or greater than 70% or 90%confidence), and the responseprovided at step 312 may provide an auditory indication of the existenceof potential additional text (e.g., a predetermined auditory tone or agenerated spoken voice indication that “additional un-translatable textexists”), which could then act as a cue to the user to view the graphicobject themselves and perhaps further attempt to decipher or otherwisematch the information.

In other embodiments wherein the inquiry is related to a street sign,license plate, or other alphanumeric text on a graphical object that theuser just passed (e.g., while in a moving vehicle), and which theelectronic digital assistant may detect that the user looked at for someminimum period of time but perhaps was unable to fully process, aresponse to an inquiry of “what was the text on that street sign I justpassed?” may be matched to objects detected within a threshold period oftime on the order of seconds (e.g., 1-30 or 1-10 seconds ago) after theinquiry was received. In such cases, the threshold period of time duringwhich the object must remain in the user's field of view for theelectronic digital assistant to consider the object absorbed may belowered beyond a default value or value assigned to other contextualsituations, and may be on the order of tenths of a second orsingle-digit integer seconds (e.g., under 10 s).

In a further particular example where the related object informationidentified at step 310 was an alphanumerical text object that was apublic safety report regarding an incident such as a police report, therelated object information may include alphanumerical text extractedfrom the police report, including portions of the police report that theuser read (e.g., spent sufficient threshold time on for the electroniccomputing device to consider it absorbed by the user, such as 10-60, or20-40, or 30 seconds) and/or portions of the police report that the userdid not read (e.g., that existed on the page but were not viewed in theuser's field-of-view as determined at step 302 and 304 or were viewed bythe user but not for the sufficient threshold time to be consideredabsorbed by the user). Using the example inquiry set forth earlier, auser search query of “what was the witnesses' name from the policereport about the WAL-STORE robbery I read yesterday?” may be matchedagainst corresponding related object information at step 310 and theelectronic computing device may identify a name of a witness from thepolice report of “John Jacobs” and provide a response at step 312 of anauditory recitation of the alphanumeric text “John Jacobs” in somemanner as already described above, or may provide the alphanumeric textback to the requesting user in any number of other manners, including atext message, e-mail, or instant message, among other possibilities.Other types of text-based reports or work assignments could be used aswell, such as a work order, a financial report, or other suchalphanumeric text-based documents.

In embodiments in which an additional second witness was included in thereport but the electronic computing device did not (or was notconfigured to) distinguish between read and unread portions of thereport, the response provided at step 312 may include both witnesses.

In embodiments in which the electronic computing device did (or wasconfigured to) distinguish between read and unread portions of thereport and the additional second witness was included in an un-readportion of the report, the electronic computing device may name thefirst witness “John Jacobs” in the response at step 312, and mayoptionally notify the user in a same or supplementary response that asecond witness (“Jack Smith”) is included in unread portions of thereport, and may name the second witness in the response as well.

In still other embodiments in which the electronic computing device did(or was configured to) distinguish between read and unread portions ofthe report and the additional second witness was included in an un-readportion of the report, the electronic computing device may alternativelyassume that the user already knows of the first witness “John Jacobs,”and instead respond with only the name of the second witness “JackSmith” that the electronic digital assistant determined that the userdid not read about yet (or looked at but not long enough to meet theminimum threshold for the electronic computing device to determine thatthe user absorbed the material) in the response at step 312, and mayoptionally notify the user in a supplementary response (automatically orupon request) that the first witness was included in the read portionsof the report that the user already read and absorbed, and may name thefirst witness in the supplementary response as well.

In additional embodiments, responses to the above or other inquiriesrelating to police reports may further include a name, location,address, time, or status extracted from the public safety report that isresponsive to the inquiry, among other possibilities.

In a further particular example where the related object informationidentified at step 310 was a graphical object, the related objectinformation may include graphical object identification information thatidentifies the graphical object, and the response to the inquiry mayinclude either or both of a graphical representation of the graphicalobject and a textual description of the graphical object. As oneexample, a user inquiry of “what was the name of the product the suspectthief took off the shelf and put in his jacket at the WAL-STORE incidentyesterday afternoon?” may be matched against corresponding relatedobject information that may include an image capture of a graphicalobject (such as a cellular phone box that was attempted to be stolen)and/or a textual description of the graphical object generated at step304 (e.g., “a cellular phone model X1X package”), and the response tothe inquiry may include a copy of the graphical object (e.g., sent asdata via a same application as the user inquiry was sent, or via a textmessage, instant message, or e-mail, or some other mechanism supportinggraphical data), and/or may include a textual description of thegraphical object provided via an auditory response in a manner such asalready described above. In embodiments where both the graphical andtextual descriptions of the object are provided in the response, theresponse may be provided via same or separate transport mechanisms.

In a further particular example where the related object informationidentified at step 310 was a graphical object capture of a human face,the related object information may include an identity of a personmatching the captured human face via a facial recognition look-up,and/or descriptive aspects thereof (such as hair color, skin color,clothing color, identified jewelry or accessories, etc.) and theresponse to the inquiry may include an image capture of the humanface,the identity of the person matching the captured human face, and/orone or more of the descriptive aspects thereof. As one example, aninquiry of “who was the person accused of stealing the product at theWAL-STORE incident yesterday afternoon?” may be matched againstcorresponding related object information that may include an imagecapture of the suspect thief and/or an identity of the suspect thief(e.g., determined via a face recognition match to a prior offenderdatabase), and the response to the inquiry may include an image copy ofthe face as a graphical object and/or a textual description of thegraphical object (e.g., the thief suspect's identity) provided via anauditory response in a same or different manner as the graphical object,either or both provided in a manner as already described above.

In any of the examples set forth above in which a graphical object isprovided in the response at step 312, the electronic computing devicemay take the further step of identifying a central frame in time fromthe video stream having a minimum level of blur, storing the centralframe (or a portion thereof including the graphical object identified inthe video stream central frame) as object information, and provide theall or the portion of the central frame including the graphical objectin the response to the inquiry.

In still other examples, and in embodiments where audio is captured at asame time as the video containing the object was captured at step 302,the response to the inquiry may include a reproduction of some or all ofthe audio actually captured at step 302, perhaps in addition to agenerated response in the manner as set forth above or in place of thegenerated response set forth above. For example, and using the exampleabove where a search query of “what was the gas station employee John'slast name that I spoke with yesterday?” and the response includes theemployee John's last name extracted from object information, the usermay further request a playback of audio reflecting the conversation withJohn at that time, and a supplementary response generated and providedto the user at step 312 may then include a copy of the audio capturedduring a same time (or within a determined period of) that the priorobject information was matched. In another embodiment where the namebadge wasn't scanned to generate the object information but the userread aloud the name badge, which was captured by the electronic digitalassistant, the response to the inquiry may include a playback of theuser originally reading the employee's name badge aloud, instead of orin place of the electronic reproduction of the voice transcription ofthe information. Other examples are possible as well.

3. Conclusion

In accordance with the foregoing, an improved device, method, and systemis disclosed for an electronic digital assistant to computationallyprocess video relative to a determined user's perception of such videoand to tailor the electronic digital assistant's inquiry response to theuser's subsequent query accordingly.

As a result of the foregoing, and in some embodiments, electronicdigital assistants computationally determine to provide or to avoidproviding information in a response to a user inquiry that theelectronic digital assistant determines the user already knows based onthe extracted user field-of-view and object recognition technicalfunctions set forth above. Accordingly, electronically generated usersearch inquiry responses can provide more individualized content to auser based on prior user detected perceptions of objects via videoanalytics occurring within a determined threshold period of time. Otherfeatures and advantages are possible as well.

In the foregoing specification, specific embodiments have beendescribed. However, one of ordinary skill in the art appreciates thatvarious modifications and changes can be made without departing from thescope of the invention as set forth in the claims below. Accordingly,the specification and figures are to be regarded in an illustrativerather than a restrictive sense, and all such modifications are intendedto be included within the scope of present teachings. The benefits,advantages, solutions to problems, and any element(s) that may cause anybenefit, advantage, or solution to occur or become more pronounced arenot to be construed as a critical, required, or essential features orelements of any or all the claims. The invention is defined solely bythe appended claims including any amendments made during the pendency ofthis application and all equivalents of those claims as issued.

Moreover in this document, relational terms such as first and second,top and bottom, and the like may be used solely to distinguish oneentity or action from another entity or action without necessarilyrequiring or implying any actual such relationship or order between suchentities or actions. The terms “comprises,” “comprising,” “has”,“having,” “includes”, “including,” “contains”, “containing” or any othervariation thereof, are intended to cover a non-exclusive inclusion, suchthat a process, method, article, or apparatus that comprises, has,includes, contains a list of elements does not include only thoseelements but may include other elements not expressly listed or inherentto such process, method, article, or apparatus. An element proceeded by“comprises . . . a”, “has . . . a”, “includes . . . a”, “contains . . .a” does not, without more constraints, preclude the existence ofadditional identical elements in the process, method, article, orapparatus that comprises, has, includes, contains the element. The terms“a” and “an” are defined as one or more unless explicitly statedotherwise herein. The terms “substantially”, “essentially”,“approximately”, “about” or any other version thereof, are defined asbeing close to as understood by one of ordinary skill in the art, and inone non-limiting embodiment the term is defined to be within 10%, inanother embodiment within 5%, in another embodiment within 1% and inanother embodiment within 0.5%. The term “coupled” as used herein isdefined as connected, although not necessarily directly and notnecessarily mechanically. A device or structure that is “configured” ina certain way is configured in at least that way, but may also beconfigured in ways that are not listed.

It will be appreciated that some embodiments may be comprised of one ormore generic or specialized processors (or “processing devices”) such asmicroprocessors, digital signal processors, customized processors andfield programmable gate arrays (FPGAs) and unique stored programinstructions (including both software and firmware) that control the oneor more processors to implement, in conjunction with certainnon-processor circuits, some, most, or all of the functions of themethod and/or apparatus described herein. Alternatively, some or allfunctions could be implemented by a state machine that has no storedprogram instructions, or in one or more application specific integratedcircuits (ASICs), in which each function or some combinations of certainof the functions are implemented as custom logic. Of course, acombination of the two approaches could be used.

Moreover, an embodiment can be implemented as a computer-readablestorage medium having computer readable code stored thereon forprogramming a computer (e.g., comprising a processor) to perform amethod as described and claimed herein. Examples of suchcomputer-readable storage mediums include, but are not limited to, ahard disk, a CD-ROM, an optical storage device, a magnetic storagedevice, a ROM (Read Only Memory), a PROM (Programmable Read OnlyMemory), an EPROM (Erasable Programmable Read Only Memory), an EEPROM(Electrically Erasable Programmable Read Only Memory) and a Flashmemory. Further, it is expected that one of ordinary skill,notwithstanding possibly significant effort and many design choicesmotivated by, for example, available time, current technology, andeconomic considerations, when guided by the concepts and principlesdisclosed herein will be readily capable of generating such softwareinstructions and programs and ICs with minimal experimentation.

The Abstract of the Disclosure is provided to allow the reader toquickly ascertain the nature of the technical disclosure. It issubmitted with the understanding that it will not be used to interpretor limit the scope or meaning of the claims. In addition, in theforegoing Detailed Description, it can be seen that various features aregrouped together in various embodiments for the purpose of streamliningthe disclosure. This method of disclosure is not to be interpreted asreflecting an intention that the claimed embodiments require morefeatures than are expressly recited in each claim. Rather, as thefollowing claims reflect, inventive subject matter lies in less than allfeatures of a single disclosed embodiment. Thus the following claims arehereby incorporated into the Detailed Description, with each claimstanding on its own as a separately claimed subject matter.

What is claimed is:
 1. A method for tailoring an electronic digitalassistant generated inquiry response as a function of previouslydetected user ingestion of related information, the method comprising:receiving, at an electronic processing device from a video capturedevice configured to track a gaze direction of a first user, a videostream including a first field-of-view substantially matching afield-of-view of the first user; identifying, by the electronicprocessing device, an object in the video stream first field-of-viewremaining in the first field-of-view for a determined threshold periodof time and, responsively, processing the object via a video processingalgorithm and causing, by the electronic processing device, objectinformation regarding the object output from the video processingalgorithm to be stored in a non-volatile store; subsequently receiving,at the electronic processing device, an inquiry for information from thefirst user; determining, by the electronic processing device, that theinquiry is related to the object information; and providing, by theelectronic processing device, a response to the inquiry as a function ofthe object information.
 2. The method of claim 1, wherein the object isone of an alphanumerical text object and a graphical object includingalphanumerical text, the object information includes alphanumerical textcorresponding to the alphanumerical text object or extracted from thegraphical object, and the response to the inquiry including at least aportion of one of the alphanumerical text itself or a transformation ofthe alphanumerical text into an audio reproduction.
 3. The method ofclaim 2, wherein the object is the alphanumerical text object and is areport regarding an incident or a work assignment order, and theresponse to the inquiry includes a name, location, address, time, orstatus extracted from the report or order and responsive to the inquiry.4. The method of claim 2, wherein the object is the graphical objectincluding alphanumerical text and is a street sign or roadsideelectronic display, and the response to the inquiry includesalphanumeric text extracted from the street sign or roadside electronicdisplay.
 5. The method of claim 1, wherein the object is a graphicalobject, the object information includes graphical object identificationinformation that identifies the graphical object by type, definition, oridentity, and the response to the inquiry includes a graphicalrepresentation of the graphical object.
 6. The method of claim 5,wherein the graphical object is a capture of a human face, the graphicalobject identification information is an identity of a person matchingthe captured human face via a facial recognition look-up, and theresponse to the inquiry includes the capture of the human face and theidentity of the person matching the captured human face.
 7. The methodof claim 5, further comprising identifying, by the electronic processingdevice, a central frame in time from the video stream having a minimumlevel of blur, and providing all or a portion of the central frameincluding the object in the response to the inquiry.
 8. The method ofclaim 1, the method further comprising storing, accompanying the objectinformation, a time and/or date at which the object first and/or lastappeared in the video stream.
 9. The method of claim 8, wherein theinquiry includes a time limitation, and the step of determining, by theelectronic processing device, that the inquiry is related to the objectinformation includes determining that the stored time and/or datematches the time limitation in the inquiry.
 10. The method of claim 8,further comprising, after a threshold period of time after the storedtime and/or date, one of deleting the object information and refrainingfrom providing a response to a subsequent inquiry from the user as afunction of the object information.
 11. The method of claim 10, whereinthe threshold period of time is a predetermined predicted time that thefirst user will independently retain information relative to the objectafter viewing the object in the first field of view.
 12. The method ofclaim 11, wherein the threshold period of time is varied based on one orboth of a measured amount of time the object remained in the first fieldof view and a measured number of repetitions in which the objectreappeared in the first field of view.
 13. The method of claim 10,wherein the threshold period of time is within a range of eight totwenty four hours.
 14. The method of claim 1, the method furthercomprising storing, accompanying the object information, a geographicallocation of the first user and/or the object, at a time at which theobject first and/or last appeared in the video stream.
 15. The method ofclaim 14, wherein the inquiry includes a location limitation, and thestep of determining, by the electronic processing device, that theinquiry is related to the object information includes determining thatthe stored geographical location matches the location limitation in theinquiry.
 16. The method of claim 1, wherein the video capture device isa user-mounted or vehicle-mounted video capture device having arelatively large field-of-view, and wherein the relatively largefield-of-view is reduced to the first field-of-view via video processingand as a function of head-tracking information or eye-gaze trackinginformation of the first user received via a corresponding head-trackingor eye-gaze tracking device.
 17. The method of claim 1, wherein thevideo capture device includes an audio capture device, the methodfurther comprising using audio captured via the audio capture device, bythe electronic computing device, to one of generate and supplement theobject information as a function of the captured audio.
 18. The methodof claim 1, wherein providing the response to the inquiry comprisescausing one of an audio and a visual output including the response tothe inquiry to be played back via one of a corresponding speaker anddisplay device associated with the first user.
 19. The method of claim1, wherein the providing the response to the inquiry as a function ofthe object information comprises refraining from including in theresponse the object or the object information, and instead, providingadditional information in the response that assumes that the first useris already aware of and has knowledge of the object and/or the objectinformation.
 20. An electronic processing device for tailoring anartificial intelligence inquiry response as a function of previouslydetected user ingestion of related information, the device comprising: amemory; a transceiver; and one or more processors configured to:receive, from a video capture device configured to track a gazedirection of a first user, a video stream including a firstfield-of-view substantially matching a field-of-view of the first user;identify an object in the video stream first field-of-view remaining inthe first field-of-view for a determined threshold period of time and,responsively, process the object via a video processing algorithm andcausing, by the electronic processing device, object informationregarding the object output from the video processing algorithm to bestored in a non-volatile store; subsequently receive an inquiry forinformation from the first user; determine that the inquiry is relatedto the object information; and provide a response to the inquiry, viaone of the transceiver, a display communicatively coupled to theelectronic computing device, or a speaker communicatively coupled to theelectronic computing device, as a function of the object information.